Data Mining for Very Busy People

F or 21st-century businesses, the problem is not accessing data but ignoring irrelevant data. Most modern businesses can electronically access mountains of data such as transactions for the past two years or the state of their assembly line. The trick is effectively using the available data. In practice, this means summarizing large data sets to find the “pearls in the dust”—that is, the data that really matters. In the data mining community, “learning the least” is an uncommon goal. Most data miners are zealous hunters seeking detailed summaries and generating extensive and lengthy descriptions. The “Data Mining and Treatment Learning” sidebar discusses some work in this area. Here, we take a different approach and assume that busy people don’t need— or can’t use—complex models. Rather, they want only the data they need to achieve the most benefits. Instead of finding extensive descriptions of things, the TAR2 “treatment learner” is a data mining tool (http://menzies.us/rx.html) that hunts for a minimal difference set between things. A list of essential differences is easier to read and understand than detailed descriptions. Overly elaborate models can complicate, not clarify, a situation. Cognitive scientists and researchers studying human decision making note that people often use simple models rather than intricate ones. Because it learns smaller models, TAR2 provides better support for realworld decision making than standard data miners.